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are known in dimension up to three, but not for general dimensions. 



Abstract 

Skorokhod problem arises in studying Reflected Brownian Motion (RBM) on an non- 
negative orthant, specifically in the context of queueing networks in the heavy traffic regime. 
One of the key problems is identifying conditions for stability of a Skorokhod problem, 
defined as the property that trajectories are attracted to the origin. The stability conditions 



In this paper we explain the fundamental difficulties encountered in trying to establish 
stability conditions for general dimensions. We prove that stability of Skorokhod problem is 
an undecidable property when the starting state is a part of the input. Namely, there does 
not exist an algorithm (a constructive procedure) for identifying stable Skorokhod problem 
in general dimensions. 



t^j- ■ Keywords: Reflected Brownian Motion, Fluid Model, Computability. 
On' 

1 Introduction 

o 

The Skorokhod problem was introduced by A. Skorokhod to model stochastic processes, typically 
diffusion processes, which are constrained to take values in a particular subset X of the Euclid- 
ian space M. d . The constraints take form of augmenting the underlying unconstrained process 
^ (diffusion) with an additional process (pushing) which is "active" only when the process is on 
the boundary of X. When the underlying process is a Brownian motion, the corresponding Sko- 
rokhod problem is often called Semi-Martingale Reflected Brownian Motion (SRBM). Solving a 
Skorokhod problem roughly refers to the problem of identifying the augmentation such that the 
modified process is well defined. A particularly important application of the Skorokhod problem is 
in the theory of queueing networks, where the state space X is usually a non-negative orthant IR+, 
the constrained process corresponds to the vector of queues appropriately scaled, and the pushing 
process corresponds to processes describing cumulative idling of servers [CYOlj , [Har90] , |Wil95] . 
In this case one can naturally construct a certain reflection matrix R such that the idling process 
Y(t) impacts the vector of queues via the matrix R. (See the next section for formal definitions 
of a Skorokhod problem, SRBM and other related notions.) 

A key question arising in the context of Skorokhod problem is stability - the property that the 
constraned process is positive recurrent. Among other things, the importance of stability stems 
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from the fact that it implies the existence of an invariant probability measure for the underlying 
stochastic process. In the context of queueing theory, one can use the invariant measure to 
obtain important insights into key performance measures of the underlying queueing network, 
such as steady state queue lengths and waiting times distributions [GZ06]. [BL09j . Unfortunately, 
stability of a Skorokhod problem turned out to be a difficult property to analyze. In the context of 
SRBM the stability becomes a property which depends on the parameters of underlying Brownian 
motion, namely the drift and the covariance matrix, and the reflection matrix R. An important 
advance was achieved by Dupuis and Williams |DW94] who connected stability of SRBM with 
the stability of the associated so-called fluid model. One can think of this fluid model as an 
SRBM with a deterministic (zero covariance matrix) Brownian motion. They showed that an 
SRBM is stable if every path in the corresponding fluid model is attracted to the origin. There 
are two issues, however, associated with this important result. First, as it was shown recently 
in Bramson [Bra] . the converse of this result is not true when d = 6. See Theorem [8] below for 
the precise statement. Second, it still leaves open the question of identifying stability conditions 
for the corresponding fluid model. Some sufficient conditions are known for restricted classes of 
the reflection matrix R |Wil95] . Additionally, a full characterization of stable fluid models as 
well as the underlying SRBM for a 3-dimensional Skorokhod problem was obtained in a series 
of papers: El Kharroubi, Ben Tahar and Yaacoubi |AEK00] . El Kharroubi, Ben Tahar and 
Yaacoubi [AEK02J . Bramson, Dai and Harrison [BDHj. However, as of now, the characterization 
of stable SRBM or stable fluid models of SRBM in general dimensions is not known. 

In this work we explain the fundamental difficulties encountered in trying to establish stability 
conditions for a fluid model of an SRBM for general dimensions. We prove that stability of a fluid 
model of an SRBM is an undecidable property when the starting state of a fluid path is a part of 
the input (see the next section for formal definitions and the precise statement). Namely, there 
does not exist an algorithm (a constructive procedure) which determines whether a reflection 
of a given linear path is attracted to the origin. We further conjecture that stability of a fluid 
model of an SRBM remains undecidable when the starting state is not part of the input. Namely, 
when stability is defined as the property that all fluid paths are attracted to the origin. This 
is stability definition used in [DW94J, and in order to distinguish it from stability of a given 
fluid path, we call it global stability of a fluid model in this paper. Likewise, we conjecture that 
stability of an SRBM is an undecidable property in general dimension d. Our result continues a 
stream of earlier works |Gam 02j . [ Gam07] . [GKR09J . where stability of constrained random walks 
in Z^. and multiclass queueing networks operating under certain classes of scheduling policies 
was shown to be undecidable. 

The concept of undecidability was introduced in the classical works of Alan Turing in 1930's 
and it is one of the principal tools for establishing limitations of certain decision problems. A 
good reference of decidability (computability) is |Sip97| . Thanks to the work of Turing we know 
that certain decision problems do not admit an effective solution in a sense of existence of an 
algorithms to solve them. We should note that undecidability property is not related to the speed 
of algorithms, or specifically, whether a polynomial time algorithm exists for a given problem. 
If a problem is shown to be undecidable, it means it does not admit any algorithm to solve it, 
no matter how slow the running time is allowed to be. There are many examples of undecidable 
problems, including Turing Halting Problem, Post Correspondence Problem, Conways' game 
of life and many others. A recent article [GS10] gives a nice overview of known undecidable 
problems in mathematics, as well as the connection of this concept with Godel's Incompleteness 
Theorem. We should note, however, that very few undecidable problems are known in the 
context of stochastic processes, probabilistic cellular automata being the only notable exception 



perhaps |GS10] . Thus we believe that this article along with |Gam02] and [GK07] contributes 
to awareness of this important notion in the community of researchers working in the area of 
stochastic processes. 

Typically one establishes undecidability of a given problem by taking a problem which is 
already known to be undecidable, and establishing a reduction from this problem to the underly- 
ing problem of interest. This is known as the reduction method. Recently several problems were 
proven to be undecidable in the area of control theory [BBK + 01j . |BT00b] . |BT00a] . In partic- 
ular the work of Blondel et al. |BBK + 01] used a device known as counter machine or counter 
automata as a reduction tool. In the present paper as in |BBK + 01j as well as in |Gam 02j . 
and [GK07] our proof technique is also based on a reduction from a Counter Machine model. In 
particular we state a known undecidable problem, namely the Halting Problem for a Counter 
Machine [HU69], and then build a reduction from the Halting Problem of a Counter Machine 
into the stability problem of a fluid model of a Skorokhod problem. If there was an algorithm to 
determine stability of a fluid model, it would imply the existence of an algorithm for solving the 
Halting Problem and this would be a contradiction. 

The remainder of the paper is organized as follows. In the next section we define the Sko- 
rokhod problem, SRBM, its fluid model and stability. Our main result, Theorem [5] is also stated 
in this section. Section [3] presents Counter Machine and the Halting Problem which is used as 
a basis of our reduction. The proof of the main result is in Sections HJE] an d El Specifically, 
a reduction from a Counter Machine to a Skorokhod problem is constructed in Section HI In 
Section [5] we show that the constructed Skorokhod problem has dynamics which mimics the one 
of the underlying Counter Machine. Finally, in Section [6] we construct a modification of the 
Skorokhod problem to connect the halting property of the underlying Counter Machine with 
stability of the Skorokhod problem. 

We close this section with some notational conventions. C([0, oo),R d ) denotes the space of 
continuous M, d valued functions defined on [0, oo). 1{-} denotes the indicator function. S(-) 
denotes the Kronecker function. Namely, S(x) = 1 for x = and S(x) =0 for any other real 
value x. All vectors are assumed to be column vectors. We use A T to denote a transposition of 
a matrix A. 

2 Skorokhod problem and stability 

Given a <i-dimensional square matrix R, the Skorokhod problem is the problem of constructing 
a map * : C([0,oo),M d ) ->■ C 2 ([0, oo),R£), such that for every x G C([0,oo),M d ) the image 
(y, z) = ty(x) satisfies the following properties 

z(t) = x{t) + Ry{t), t e K+ (1) 

y(0) = 0, yj(t) is non-decreasing for all j = 1, . . . , d (2) 

Zj(s)dyj(s) = 0, j = l,...,d. (3) 

where the integral in ([3]) is in Stieltjes sense, which is well defined since yj are non-decreasing. 
Intuitively, the meaning of the constraint fl3]) is that the process yj can increase only at times 
when Zj = 0. We say that yj is active in the time interval (si,S2) if Vj is strictly increasing in 
this interval. We also say that yj is active at the unit rate if it increases at the unit rate. Namely 
Vj(s) —yj(si) = s — si for all s e (s\, S2). We say that yj is passive over (s\, S2) iiyjfa) = yj{si)- 



The specific Skorokhod problem we will construct in this paper will have many variables active 
at the unit rate over various time intervals. 

The existence of such a $ is completely and nicely characterized by the matrix property 
known as completely-iS property A matrix R is defined to be 5-matrix if there exists a d- vector 
w > 0, such that Rw > in a coordinate-wise sense. A matrix R is completely-5 if every 
principal submatrix of R is a 5-matrix. (A principal submatrix is the one where the indices of 
rows and columns are the same). In particular every completely-5 has positive diagonal elements. 
The following result is established in a series of papers [AB89J . [AB91] . JTW93] . 

Theorem 1. A Skorokhod map \& exists if and only if the matrix R is completely-S. 

Two special cases of Skorokhod problem are of particular importance. The first one cor- 
responds to the case when x(t) is a stochastic process, specifically a Brownian motion with a 
starting state z G K.1, drift vector 9 G M. d and covariance matrix E, typically assumed to be non- 
singular. This special case is usually called Semi-Martingale Reflected Brownian Motion (SRBM) 
and arises in the context of heavy traffic theory of queueing networks |Wil95] .|UY01j. An SRBM 
is thus completely specified by data (zo, $? E, R). It shown in Taylor and Williams |TW93j that, 
though the Skorokhod mapping \l/ may not be unique for some completely-iS matrices R, in 
the context of SRBM it is unique in law. Namely, the distribution of x(t),y(t),z(t) is uniquely 
defined by (z , 9, E, R). An SRBM is defined to be stable if it is positive recurrent, in which case 
there exists a unique time invariant distribution. The stability property does not depend on the 
starting state zo, and thus is a property of the triplet (9, E, R). A key outstanding open problem 
is determining when is a triplet (9, E, R) stable. A significant partial progress is obtained by 
considering the so-called fluid models or fluid paths in a Skorokhod problem, and this is our 
second important special case. 

Given a vector z G M.+ and a vector 9 G M. d , consider the linear function x(t) = z + 9t, 
and the corresponding (set of) solution(s) (y(t),z(t)) = ^/(x(t)). The triplet (x(t),y(t), z(t)) 
is called called a fluid path for the reason discussed below. The system (z , 9, R) or (9, R) is 
called a fluid model of an SRBM. One can think of a fluid model as an SRBM with a degenerate 
(deterministic) Brownian motion input function x(t). 

Definition 1. A fluid model (zo,9,R) is defined to be stable if every solution (y(t),z(t)) = 
ty(x(t)) of the Skorokhod problem with x(t) = zo + 9t has property lim^oo z(t) = 0. A fluid model 
(9, R) is defined to be globally stable if it is stable for every starting state zq. 

The importance of this definition stems from the following result established in Dupuis and 
Williams |DW94j . 

Theorem 2. Suppose (9, R) is globally stable. Then an SRBM (9, E, R) is stable for every 
non-singular covariance matrix E. 

The proof of this result is based on the fluid rescaling (functional law of large numbers) 
technique and thus justifies the terms "fluid model" and "fluid paths". One would naturally 
hope for a converse result, thus showing that stability of SRBM is completely determined by the 
stability of its fluid model (9, R). Unfortunately, this hope did not materialize for six- dimensional 
SRBMs. The following result was recently established by Bramson [BraJ. 

Theorem 3. There exists 9 G K-+,E, R G M 6x6 such that the fluid model {9,R) is not globally 
stable, but the underlying SRBM (9, E, R) is positive recurrent. 



On a positive side at least for the case d < 3 the equivalence does take place. Moreover, in 
this case the stability of SRBM and the global stability of the corresponding luid model can be 
constructively characterized in terms of (9, R). The stability for the case d — 1 is given simply by 
9 < and was known for a while. The case d = 2 was resolved by El Kharroubi, Ben Tahar and 
Yaacoubi |AEK00] . In a later paper El Kharroubi, Ben Tahar and Yaacoubi |AEK02] identified 
exact conditions for global stability of a fluid model when d = 3. The link with an SRBM when 
d = 3 was resolved only recently in Bramson, Dai and Harrison [BDHJ. 

Theorem 4. Suppose d < 3. Then SRBM (9, S, R) is positive recurrent if and only if the fluid 
model (9, R) is globally stable. Moreover, the stability of the fluid model (9, R) can be verified by 
checking a system of equalities and inequalities. 

The second part of the theorem needs elaboration, but instead of explicitly giving the set of 
global stability conditions we simply refer the reader to [BDHJ. For us the only relevant fact is 
that the stability condition can be constructively verified by checking a series of equations and 
inequalities in a rather straightforward way. In light of this positive result, one would hope to 
extend this result for general d. We can summarize this question as well as the state of the art 
as follows. 

(a) When is an SRBM (9, S, R) positive recurrent? The answer is known for d — 1,2, 3, but is 
unknown for general d. 

(b) When is a fluid model (9,R) globally stable? The answer is known for d = 1,2,3 and 
coincides with the answer for (a) for every non-singular S, but is unknown for general d. 

(c) When is a fluid model (zo, 9, R) stable? 

In this paper we resolve question (c), albeit in a somewhat unexpected way. We establish 
that this problem is undecidable and this is the main result of this paper. 

Theorem 5. The property "(zq,9,R) is stable" is algorithmically undecidable (non- computable) . 
Namely, there does not exist an algorithm which given an arbitrary input zq G R+, 9 e M. d and a 
completely-S matrix R outputs YES, if the fluid model (zo, 9, R) is stable and NO otherwise. 

While we could only establish this result for problem (c), we conjecture that all problems 
(a)-(c) are undecidable. The difficulty of establishing undecidability of the (b) problem is as fol- 
lows. While there is a well defined notion of a "global" halting property of a Counter Machine, 
unfortunately this does not appear to imply undecidability of the global stability of (9, R) prop- 
erty, since in our reduction there are starting states zq and the corresponding trajectory which 
does not correspond to any trajectory of the Counter Machine. In other words the mapping 
T — > (zq, 9, R) is not necessarily one-to-one as far as trajectories are concerned. Without analyz- 
ing the trajectories not corresponding to the trajectories of the underlying Counter Machine it 
does not appear to be possible to use the reduction tool. 

Proving undecidability of (a) also seems problematic since we need to simulate deterministic 
trajectories of a Counter Machine with stochastic trajectories of an SRBM. In order to prove 
undecidability of the positive recurrence of SRBM, it appears that one would need to construct 
an SRBM in which we can "control" stochastic trajectories for an infinite amount of time. At 
the present moment we are not capable of doing this. 

Theorem [5] applies to Skorokhod problem in general dimension d. It would be natural to 
conjecture that the stability of (zq, 9, R) problem is undecidable for some fixed dimension d, for 



example when d = 6 - the dimension in which the counterexample of Theorem [8] takes place. 
In our reduction the dimension d of the Skorokhod problem depends directly on the number of 
states (see the next section) of the underlying Counter Machine. It turns out that the Halting 
problem of a Counter Machine is undecidable only when the number of states is general, since 
there exist only finitely many Counter Machines with a given number of states. It is known that 
any problem with a finite number of instances is decidable. This, however does not imply the 
same for the Skorokhod problem, since the number of Skorokhod problems instances (zq, 9, R) is 
infinite even for a fixed dimension d, since there is an infinite set of possibilities for the parameters 
zq, 9 and R even for a fixed dimension d. All this means is that our reduction technique does not 
lead to the instability result for the case of a fixed dimension d. We conjecture that there exists 
a large enough, but dimension d (perhaps d = 6) in which the stability of a fluid model (zq, 9, R) 
is undecidable. 

3 Counter Machine, Halting Problem and undecidability 

A Counter Machine which we define below is a deterministic computing machine which is a 
simplified version of a Turing Machine - a formal description of an algorithm performing a 
certain computational task or solving a certain decision problem. (For a definition of a Turing 
Machine see |Sip97|). We now define a Counter Machine and the Counter Machine Halting 
Problem (CMHP) which is known to be undecidable. Our main technical result is the reduction 
of CMHP into the stability of a Skorokhod problem (zq,9,R) such that the Counter Machine 
halts if and only if (zq,9,R) halts. 

A Counter Machine is described by 2 counters Ri,R 2 and a finite collection of states S = 
{1,2, ... ,m}. Each counter Ri contains some nonnegative integer in its register. Depending 
on the current state i G S and depending on whether the content of the registers is positive 
or zero, the Counter Machine is updated as follows: the current state i is updated to a new 
state j G S and one of the counters has its number in the register incremented by one, decre- 
mented by one or no change in the counters occurs. More specifically, a Counter Machine 
is a pair (S = {1, . . . ,m},T). where T is configuration update function T : S x {0, l} 2 — > 
S x {(—1, 0), (0, —1), (0, 0), (1, 0), (0, 1)}. A configuration of a Counter Machine is an arbitrary 
triplet (i, Ci,C 2 ) £5x T? + . A configuration (i,Ci,C 2 ) is updated to a configuration (i, C[,C' 2 ) 
as follows. Given a configuration (i, Ci,C 2 ) suppose Y{i,l{C\ > 0}, 1{C 2 > 0}) = (i',1,0). 
Then the current state is changed from i to i' , the content of the first counter is incremented 
by one and the second counter does not change: C[ = Ci + 1,C 2 = C 2 . We will also write 
T : (i,Ci,C 2 ) -> (i'jCi + 1,C 2 ). Suppose, on the other hand, T(i,l{d > 0}, 1{C 2 > 0}) = 
(i', (—1, 0)). Then the current state becomes i', C[ = C\ — 1, C 2 = C 2 . Similarly, if T(i, 1{C\ > 
0}, 1{C 2 > 0}) = (i', (0, 1)) or r(z, l{Ci > 0}, 1{C 2 > 0}) = {i' , (0, -1)), the new configuration 
becomes (i, d, C 2 + 1) or («', C x , C 2 - 1), respectively. If T(i, l{C t > 0}, 1{C 2 > 0}) = (i', (0, 0)) 
then the state is updated to i', but the contents of the counters do not change. It is assumed 
that the configuration update function T is consistent in the sense that it never attempts to 
decrement a counter which is equal to zero. The present definition of a Counter Machine can be 
extended to the one which incorporates more than two counters, but such an extension is not 
necessary for our purposes. 

Given an initial configuration (i°, z\, z®) € S x 7?, the Counter Machine uniquely determines 
the subsequent configurations (i 1 , z\, z\), {i 2 , z\, z 2 ), ■ ■ ■ , (i*, z\,z t 2 ),... . We fix a certain configu- 
ration (i* ,C{,C 2 ) € S x 2? + and call it the halting configuration. If this configuration is reached 
then the process halts and no additional updates are executed. The following theorem establishes 



the undecidability (also called non-computability) of the halting property. It is a classical result 
and can be found in |Hoo66] . 

Theorem 6. Given a Counter Machine (S,T), initial configuration (i ,^,^) and the halting 
configuration (i*, Cj", C2), the problem of determining whether the halting configuration is reached 
in finite time (the Halting Problem) is undecidable. Without the loss of generality it may be 
assumed that C{ = C| = 0. 

We will need to following small modification of the theorem above. 

Theorem 7. Given a Counter Machine (S, T), initial configuration (i ,^,^) and a state i* , 
the problem of determining whether the state i* is reached in finite time is undecidable. Moreover, 
without the loss of generality it may be assumed that if and when the state i* is reached, we also 
have C\ — Ci — 1. 

Namely this theorem states that determining whether a given state (vs configuration) is 
reached is undecidable as well, and if it is reached the counters take particular values (unity). 
We call i* the halting state. 

Proof. The proof is a simple reduction from the CMHT. Given a Counter Machine with the 
halting configuration (i*, 0, 0), augment the set of states by a new state m + 1. Define T(i*, 0, 0) = 
(m + 1, 1, 1) (regardless of what it was before) and leave T(i*, C\, C2) intact for all other C\, C<i- 
Note that we have updated only the halting configuration. Define T(m + 1, -, •) arbitrarily. Notice 
that state m + 1 is reached in the augmented Counter Machine if and only if the original Counter 
Machine halts starting from (i°, C°, C®)- Notice also that by our augmented rule if m + 1 is 
reached at time t, then at this time both counters have values equal to unity. □ 

4 A reduction of a Counter Machine into a Skorokhod 
problem 

Before we present the formal proof of our main result, Theorem we describe a high level idea 
behind the proof. In this and the following Section \5\ we consider an arbitrary Counter Machine 
with m states and construct a Skorokhod problem with the following properties. The dimension 
of the Skorokhod problem is d — 5m + 9. Two of the coordinates of the z(t) vector in the 
Skorokhod problem will be "responsible" for the value of the counters in the Counter Machine, 
and m other coordinates of z(t) will be "responsible" for the state of the Counter Machine. The 
remaining coordinates serve auxiliary roles. Specifically, the construction will be such that if 
the counter machine has counter values C\ and Ci and is in state % at time t, then coordinates 
of z corresponding to counters have also values C\ and C2 at time 5t, and the m coordinates 
corresponding to the state have all values 1 except for the coordinate corresponding to the state 
i, which will have value zero at time 5t. The time interval [5t, 5t + 5] will correspond to emulating 
the one step t — > t + 1 of the Counter Machine. Thus we can "read off" the state of the Counter 
Machine from the state z(t) of the Skorokhod problem. This by itself does not prove Theorem [5] 
since, as it turns out the Skorokhod problem constructed this way is never stable, in the sense 
that it never enters the zero state. The completion of the proof is presented in Section [6] where 
we construct a small modification of our Skorokhod problem with the following property: if 
the Counter Machine enters the halting state i* at some time T, then our modified Skorokhod 
problem enters zero state at time 5T + 1. Alternatively, if the Counter Machine never enters 



state i*, then our Skrokhod problem never enters the zero state. Additionally, we show that 
the R matrix corresponding to our modified Skorokhod problem is completely-iS. From this 
construction we conclude that if we had an algorithm for checking whether a Skorokhod problem 
(zq, 6, R) with a completely-iS matrix R is stable, we would also have an algorithm for checking 
whether a Counter Machine enters its halting state i*, and thus obtain a contradiction. 

We now present details of the construction and the proof of Theorem |5j Consider a Counter 
Machine with states 1, . . . , m and starting configuration (i°, C°, (7°). We now construct a Sko- 
rokhod problem ([I]),©,©- x,y,z have dimension d = 5m + 9. x(t) = z — 9t,9 G Z^ where 
9i = — 1, 1 < % < 5 and d\ = otherwise, and zq will be specified later. Thus 

-1 
-1 
-1 
-1 




(4) 



V o J 

We now describe matrix R e Z dxd . The rows and columns of R are grouped according to 
different roles they will be playing in the construction. For convenience the groups are denoted 
by A, B, C, D, E, F, where the sizes are 5, m, 2, 2, 4m and 4m respectively. Specifically the first 
five rows and columns belong to the group A, the next m rows and columns belong to the group 
B, the following two rows and columns are group C, etc. The submatrix consisting of rows in 
one group and columns in some possibly other group will be denoted by a concatenation of the 
corresponding letters. So, for example the 5x5 submatrix AA consists of rows and columns in 
A, m x 4m submatrix BE consists of rows in B and columns in E, etc. Several submatrices will 
be equal to zero and will be denoted by 0. Several other square submatrices are identity matrices 
and are denoted by /. In particular, our matrix R is assumed to have the following structure: 



R 



( AA 
BA 
CA 
DA 




I 








I 
I 



\ 

BE 

1 CE 
7 



(5) 



EA EB EC I I 
\ FA FB FC I I J 

Let us now describe the remaining submatrices of the matrix R. AA is set to be 



AA 



( 1 2 1 1 \ 
12 11 
10 12 1 
110 12 

\ 2 1 1 1 ) 



(6) 



Namely AA is a row vector [1, 2, 1, 1, 0] rotated five times. From now on we adopt a notation of 
the form XYij where X, Y are one of the matrices A, B, . . . , F and i and j range over indices of 
X and Y respectively. So for example AA2^ is the entry (2, 3) in the submatrix AA. Namely, 
^4^2,3 = 2. We will also use notation of the form XYj* to indicate a row vector corresponding 



8 



to index % in the submatrix XY. Similarly, we will us XY^j, or R^, R*j when the underlying 
matrix is entire R. 

Let us describe the m x 5 matrix BA. We set BA^\ = — 1 and BAij = for 2 < j < 5 for 
all 1 < i < m. As for the matrix BE, for every 1 < i,j < m and every b, c G {0, 1}, define 
BEj^ i+ 2b+ c = if T(i, 6, c) = (j, Ai, A 2 ) for some updates Ai, A 2 G { — 1, 0, 1} and = 1 otherwise, 
where V is the configuration update function of the underlying Counter Machine. In the matrix 
form we have 



BA 



( -1 \ 
-10 

y -1 J 



BE 



' BE\ i BE\ 2 
BE 2 a BE 2 7 



BEi^ m 
BE 2 A m 



\ 



V 



BE. 



m,l 



BE. 



m,2 



BE n 



(7) 



,4m / 



Matrices CA and CE are 2x5 and 2 x 4m respectively. For i = 1,2 define CA iA = CA iA = - 
and CAij = otherwise. Also define C E ljii+2b+c = A x + 1, C E 2 ^ i+2h+c = A 2 + 1 if T(i, b, c) 



(j,Ai,A 2 ). Observe that CE lAi+2b+c , CE 2Ai+2b+c > 0. Thus 



CE\ t i CE\ )2 
CE 2 i CE 22 



CEi^ m 
CE 2 A m 



Matrix DA is 2 x 5. For i = 1, 2 define DA i)2 

DA = 



1 and DA,- ,• = otherwise. Thus: 



H,J 



0-1000 
0-1000 

Matrices £M, -E.B and EC are 4m x 5, 4m x m and 4m x 2 respectively. Define EAu+2b+c,\ 
c, EA Ai+2b+Cy3 = -1 and EA 4i+2fe+C)i = for j = 2, 4, 5. Define EB M+2 b+ c ,i = -1, EB Ai+2b+ 
for j ^ i, 1 <i,j < m. Finally, define -EC^+^+^i = 26—1, -EC^+^+c^ = 2c — 1. Namely 

/ 



(9) 



(■■j 



EA 



\ 



EA 



4m, 1 



0-100 



EB 



-1 


. 


. 


-1 


. 


. 


-1 


. 


. 


-1 


. 


. 





. 


. -1 





. 


. -1 





. 


. -1 





. 


. -1 



EC 



( EA hl 0-1 0\ -10... / EC 1A EC li2 \ 

\ EC Ami i EC \ m .2 J 

(10) 

Finally, matrices FA,FB and FC are 4m x 5,4m x m and 4m x 2 respectively. We set 
FB = EB,FC = EC. Define FA 4i+2b+cA = -b - c,FA 4i+2b+c , 3 = -1, FA 4i+2b+cA = -4 

and FA 4i+ 2 b+c ,5 = 4: 



/ FA hl 



FA 



\ 



FA 



4m, 1 







•1 -4 4 \ 
/ 



-4 4 



/ FC ltl FCi 2 \ 



FC 



\ 



FC 



4m, 1 



FC 



:n) 



4m 



'/ 



This completes the description of the i? matrix. A natural question is whether R is completely-5. 
We defer this question to Section [6] where we first modify R slightly and then show that indeed 
its modification is completely-5. 
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5 Dynamics of the Skorokhod problem 

Consistent with our notations A,B,...,F, the coordinates of 6 and the processes x(t),y(t), z(t) 
are grouped by A, B, . . . , F as well. Again we will use the notation of the form lx,X = 
A,B, ... ,F to denote a portion of a vector I corresponding to the group X. Let lx,% denote the i-th 
entry corresponding to the portion X. We will also sometimes use lx,* in place of lx ■ Specifically, 
as per construction in the previous section 6 a = 6a,* = (— 1, • • • , — 1) T , 6 B = 0, . . . , 6p = 0. Equiv- 
alently 6 A ,i = ... = 6 A ,5 = -1, 6 B ,i = ■■■ = d B , m = 0, etc. Also 6 X = for X = B, C, D, E, F. 

Now let us define the initial state x(0) = z(0) = zq of the process x(t). Suppose the config- 
uration of the Counter Machine at timet = is (i°,Ci,C 2 ). Then we set £a,i(0) = £,4,5(0) = 
0, £,4,2(0) = £4,3(0) = £4,4(0) = 1, £^0(0) = and £bj(0) = 1 for all j 7^ i°, 1 < j < m. We set 
£ C ,i(b) = ClxcM = C° 2 . x D>1 {0) = xn, 2 (0) = 0. Finally x EAi+2b+c (0) = 3,x FAi+2b+c (0) = 4 
for all 1 < i < m,b,c,€i {0, 1}. Namely, in the vector form 



x A (0) 



£f(0) 



/0\ 

1 
1 
1 



/4\ 

4 

\ 4 / 



£b(0) 



1 

1 
1 

v 1 y 



£c(0) 



c° 2 



x D (0) 



xe{0) 



/3\ 
3 

V 3 / 



121 



The next theorem is our key technical result. It shows that the configuration of a Counter 

can be encoded by a Skorokhod problem just constructed at times 



Machine at times t 
5£, t 



0,1, 



0,1,2, 



Theorem 8. Given a Counter Machine (S, T) with a starting configuration (i°, C°, C 2 ) suppose 
its configuration at time t is (i l , C{, C|), t = 0, 1, . . . . Then at time 5t the state z(5t) of the 
corresponding Skorokhod problem is as follows: ZA,i(5t) = ZA^(5t) = 0,z J 4 j2 (5t) = za,3(5£) = 
ZAA(5t) = 1, zs,it(5t) = and ZB,i(5t) = 1 for all i 7^ i l , zc\k(5t) = C l k1 k = 1,2, zo,*(5t) = 
0,ZE t *(5t) = 3,ZF,*(5t) = 4. In particular, zb,* and zc,* encode the state i* and the counters 
C\,C 2 of the Counter Machine at time 5t, respectively. 

In vector form the claimed state z(5t) is thus described in ffT2|) . where x replaces z, the unique 
in the I?-component of z corresponds to index i f , and C{, C 2 replace C°, C 2 . 

By itself this theorem does not prove our main result Theorem it is easy to see that z(t) 7^ 
for all t. Namely, our Skorokhod problem is not stable. In Section E] we construct a modification 
of R such that the halting state i* is reached if and only if the Skorokhod problem is stable. 
Theorem |5] then will follow from Theorem [71 

Proof. The remainder of this section is devoted to the proof of Theorem |HJ The argument is based 
on induction. The base case is covered by our assumptions on the initial state £(0), 2/(0), z(0). For 
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the induction part we simply carefully track the dynamics of the Skorokhod problem over a time 
interval [5£, 5t + 5] and show that if at time 5t the Skorokhod problem encodes the configuration 
of the Counter Machine at time t, then it will also do so at the time 5t + 5 for the configuration 
at time t + 1. 

We now provide details of the induction argument. We begin by separately analyzing the 
dynamics of Za {t). Then we analyze the dynamics of the remaining part of the process z(t) over 
intervals [5£, 5£ + 1], . . . , [5£ + 4, 5£ + 5] separately. 

5.1 Dynamics of ZA{t) 

As we now show the dynamics of Za,*, Va,* is periodic, does not depend on the dynamics of the 
Counter Machine and does not depend on the dynamics of the rest of the vectors z, y, in some 
appropriate sense. 

Lemma 1. For every t G Z + andi G {1,2,3,4}, ZA,i(5t+i) = ZA,i+i(5t+i) = and ZAj{5t+i) = 
1 for all j ^ i,i + 1. Also ZA,i{5t) = ZA,5{5t) = 0, and ZA,j{5t) = 1 for all j ^ 1,5. Namely, 
vector ZA{t) cycles through the following five vectors over the time instances 5t, 5t + 1, . . . , 5t + 4: 

/0\ /0\ /1\ /1\ /1\ 





1 
V 1 / 





1 
1 

V 1 / 



i 




V 1 / 



i 
i 





(13) 



1 
1 
1 

\o/ 

Moreover, for every t G Z + and i = 0,1,2,3,4, variable yA,i+i is active at the unit rate in 
(5t + i,5t + i + l), and is passive otherwise. In particular, during the time period (5t + i, 5t + i + 
1), < i < 4, the variable yA,i+i is the unique active variable among five variables yA,*- 

Proof. Observe that no variables other than yA,* influence Za,*- Thus it suffices to establish the 
result for t = and demonstrate the periodicity property za(5) = za(0). First consider time 
period [0, 1]. Recall that za,i = za,5 = and za,% = 1 for i ^ 1,5. From ([1]) we have for every 

8 6(0,1) 

za,2(s) = ^4,2(0) - 9a,2S + Va,2(s) - ^(O) + 2(y A ,3(s) - ?/a,3(0)) + VA,i( s ) - VaAO) 
+ Va,5(s) -Va,5(0) 
>l-s 
>0, 

where the non- decreasing property ([2]) of yj was used. The constraint ()3]) then implies that yA,2 
remains passive in the interval (0,1). Similarly we show that variables yA,3 and yA,4 remain 
passive over the same time interval. Applying again (JTJ) we obtain for every s G (0, 1) 

za,i (s) = - s + yA,i(s) - Va,i{0), 

zaM = - s + 2(y A ,i{s) - yA,i(0)) + VaM - 2M,s(0). 

By 02]) and non-negativity of z, the first identity implies that Va,i(s) = s. Applying this to 
the second identity we obtain that za,5 becomes strictly positive for s > and therefore the 
variable yA,5 is passive over the interval (0,1). We conclude that for s G (0,1), za,2(s) = 
1 — s, za,3(s) = 1 — s + s = 1, -2^4(5) = 1 — s + s — 1, za,5(s) = — s + 2s. In particular, 
z a,i(X) = ^,2(1) = 0,2:^3(1) = za^{X) = 2,4,5(1) = 1- This establishes the claim for s G (0,1). 
The proof for the cases s G (1, 2), (2, 3), (3, 4), (4, 5) is obtained by shifting the indices by one. □ 
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5.2 Interval [5t, 5t + 1] 



In this and the remaining subsections we will say that a variable i/j is active when it is active at 
the unit rate. When a variable is active at a non-unit rate we will explicitly say this. For every 
i = 1, 2, . . . , m and 6, c = 0, 1 let 



T(i, b, c) = -b - c + (26 - l)8(C[ = 0) + (2c - 1)<5(C* = 0) - 5(i 



(14) 



Lemma 2. In the time period (5£, 5£ + 1) variables yA,i,yB,it ore active. Variable yc,i-,i = 1,2 
zs active iff Cj = 0. ^1// i/ie remaining y variables are passive. The state z(5t + 1) is as follows: 
z B> ,(5t + 1) = 0, z Cti (5t + 1) = C\ - 1 + 8(Cf = 0), i = 1, 2, z Dii (5t + 1) = <J(C? = 0), i = 1, 2, 
ZE,4i+2b+c(5t + 1) = 3 + T(i, 6, c) for alii = 1, . . . , m, and zf,4j+26+c(5£ + 1) = 4 + T(i, 6, c) /or a// 
z = 1, . . . , m and b, c = 0, 1. Namely, in the vector form 



z B (5t+l) 



z E (5t + 1) 



z F (5t + 1) 



/0\ 




z c {ht + 1) 

\ 
/ 



C{-1 + 5(C{ = 0) 
01-1 + 6(01 = 0) 



z D {5t + l) 



S(C{ = 0) 
8(C\ = 0) 



3 + T(i, 6, c 

V 

/ ; \ 

4 + T(z,6,c 



V 



/ 



Proof. We begin by identifying the active variables. Among y^i we already established in 
Lemma LT] that only y^,i is active. Consider now variables ys,i for i ^ i l . Since by our inductive 
assumption ZB,i{5t) = 1, since BA^i = — 1 and all the remaining entries of rows corresponding to 
B are non-negative, then zb,% remain positive in (5£, 5t+ 1) and therefore, all ys,i remain passive 
during this time interval. As for the variable zb,& which equals zero at time t = 0, we have for 

se(0,l) 

ZB,#(5t + s) = -(y A ,i(5t + s) - yA,i{5t)) + (|/b,j*(5£ + s) - y B ,it(5t)) 

+ J] S^^BJ^ + S)-^,^)). 

l<j<4m 

Since -B.E > 0, and the rate of y^,i is unity, we see yBi*(s) increases at the rate at most unity. 
Later on we will show that in fact y E ,* remain passive and therefore ys^is) increases precisely 
at the unit rate. 

Consider variables yc,i,i = 1,2. We have for s G (0, 1) by our inductive assumption 



*c,i( 5t + s ) = c l~ (yA,i( 5t + s )~ VA,i{5t)) + {y c ,i(5t + s)- y c ,i(5t)) 
+ {VD,i{5t + s) - y D ,i{5t)) 

+ J2 CE iAvEA 5t + s )-yEA 5t ))- 



(15) 
(16) 



l<j"<4m 
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Recall that the entries of CE are non-negative. The rate of y A ,i is unity. If C\ > 1, this implies 
that variables zc,i remains positive on [5£, 5t + 1) and therefore yc,% is passive. If C\ = 0, then, 
as in the case of y B ,i* variable, yc,i can be active at the rate at most unity. When we show that 
y B ,* and He,* are passive, this will imply that yc,i is active at the unit rate. 

Consider variables yD,%- The only negative entries in rows corresponding to D are DAi }2 - 
However, y A>2 is passive. Thus yo,i, i = 1,2 remain passive. 

Consider variables yE,*- We have established that y B)i are passive for i ^ i*, y c ,t are passive 
unless C\ = and y^,* are passive. The entries of EE and EF are non-negative. Therefore 

z E ,u+2b+c(5t + s)>3-{b + c){y AA {5t + s) - y A ,i( 5t )) ( 17 ) 

- 5(i = i*)(y B ,;t(5i + s)- y B ,A 5t )) 
+ 8{C[ = 0)(26 - l)(2/c,i(5t + s)- yc,i(5t)) 
+ <S(C* = 0)(2c - l)(3/ c , 2 (5t + s) - yc, 2 (5t)). 

Let us verify case by case that ZE,a+2b+c remains positive in [5t, 5t + 1). We know that y A ,i 
increases at the unit rate. We have already established that ysfi, Vc,% increase at the rate at most 
unity. If b = c = 0, giving 26—1 = 2c— 1 = —1 we obtain z E ,4i + 2b+c(5t+s) > 3 — (l + l + l)s > 0. If 
b = 1, c = 0, we again obtain ZE,a+2b+c(5t + s) > 3 — (1 + 1 + l)s > 0. Similarly when b = 0, c = 1. 
Finally b = c = 1, we also obtain ZE,-a+2b+c(5t + s) > 3 — 3s > 0. We have established that 
ZE,4i+2b+c remains positive. This implies that yE,* remain passive. This further implies that ysfi 
is active at unit rate and yc,i is also active at unit rate if C\ = 0. 

The case of yp,* variables is similar to the one of yE,* in light of the fact that at time zero 
these variables have value 4 as opposed variables yE,* which have value 3. We conclude that the 
only active variables are y A) i, ys,* and yc t i when C\ = 0, and they increase at a unit rate. 

It is now easy to identify the state z(bt+l). For s G (0, 1) we have ZB,i{^t+s) = l — (y A) x(5t+ 
s)-yA,i(5t)) = 1-sforz ^ i f ; z Byi t(ht+s) = 0-(yA,i(5t+s)-y A ,i(5t)) + (yB,it(5t+s)-y B ,it(5t)) = 
0. Therefore zs t *(5t + 1) = 0. We may rewrite ( TT5l) as 

zcA^t + s) = CJ-s + {ycA^t + s)- y c ,i(5t)). 

We obtain zc% = C\ — 1 when C\ > 1 and = otherwise. We can write this as za = G\ — 1 + 
6(Cj = 0). 

For z B ,* variables we have 

ZD,i(5t + s) = yc,i(5t + s) - yc,i(5t). 

Thus ZD,i(5t+ 1) = 1 if yc,i is active, namely if C\ = and = otherwise. In short, 2d,»(5£ + 1.) = 
5{C\ = 0). Rewrite (JTZD as 

ZEAi+2b+c(5t + s) = 3-(b + c)s- 8{i = i l )s + 8[C[ = 0)(26 - l)a + 5(C\ = 0)(2c - l)s. 

Evaluating at s = 1 we obtain ZE,*(5t + 1) = 3 + T(z, b, c). 
Finally, we have 

2F,4 i+ 2fe+ c (5t + s) = 4 - (6 + c)s - 8{i = i l )s + 5(C{ = 0)(26 - l)s + 5(C l 2 = 0)(2c - l)s. 

Evaluating at s = 1 we obtain z F ,*(5t + 1) = 4 + T(z, b, c). D 
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5.3 Interval [5t + 1, 5t + 2] 

We will show that the only change occurring in this time interval is increasing the values of 
zc,i,zc,2 by one. 

Lemma 3. In the time interval (5£ + l,5£ + 2) variable y&,2 is active. yo,i is active if and only if 
C\ > for i = 1, 2. All the remaining y variables are passive. Moreover, zc,i{5t + 2) = Cf, i = 
1, 2, and z x ,*(5t + 2) = z x ^(5t + 1) for X = A,B, D, E, F. 

Proof. The only negative entries in R in rows corresponding to B are BA\^. By Lemma[H|/A,i 
is passive. Thus variables ys,* remain passive. The same applies to variables yc,*- We skip the 
analysis of yo,* for a moment. Observe that yE,*,VF,* remain passive since by Lemma [U|/a,i and 
yA,3 are passive, and as we have established above, ys,* are passive. It remains to analyze yo,*- 
We obtain for s G (0, 1) 

z Dti (5t + l + s) = z D:i (5t + 1) - (2/A, 2 (5t + 1 + s) - y A ,2(5t + 1)) + (y D ,i(5t +! + «)- VD&t + 1)) 
= 5{C\ = 0) - s + (y Dti (5t + l + s)- y Dti (5t + 1)), 

which immediately implies that y^^ is active if C\ > and passive otherwise. 

It is easy now to identify the state at time 5t + 2. By direct inspection we obtain that for 

se(o,i) 

z c ,i(5t + l + 8 ) = z c ,i(5t + 1) + y D ,i( 5t + 1 + s) - VdA^ + l ) 
= C\ - 1 + 8{Cl = 0) + S 5(C< > 0) 
= CJ. 

All the remaining variables z* remain the same. This concludes the proof. □ 

5.4 Interval [5t + 2,5t + 3] 

Lemma 4. In £/je ttme interval (5t + 2, 5t + 3) t/ie variable yA.,3 is active and variable yE,a t +2b+c 
is active for b = min(C*, 1), c = min(C2, 1). All the remaining variables are passive. Moreover, 
at time 5t + 3 the state z(5t + 3) is as follows: z B>i t+i(5t + 3) = and ZB,i{5t + 3) = 1 fori ^ i t+l . 
z c ,i(ht + 3) = Cj + Ai + l,i = 1,2. z Dti (5t + 3) = 0, i = 1,2. Finally, z EAi+2b+c (5t + 3) = 
3+T(z,6,c) + l{(z,fo,c) = («',&',<:')} and z F>M+2b+c (5t+3) = 4+T(z,6,c) + l{(z,6,c) = (z*,6*,c*)}. 
Namely, in the vector form 
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z B (5t + 3) 



z c {5t + 3) 



C\ + Ai + 1 
C\ + A 2 + 1 



z E (5t + 3) 



z F (5t + 3) 



1 

1 

1 

v i y 

/ 

2 + r(z,6,c)) + l{(z,6,c) = (^V)} 

V 

/ \ 

3 + T(i, 6, c) + l{(i, 6, c) = (i*, 6 4 , c*)} 



2 D (5t + 3) 



V 



/ 



Proof. By non-negativity of -Bi? and CE sub-matrices and since i/a,3 is the only active among A 
variables, we verify directly that ys,*, yc,*, Vd,* remain passive. Now consider variables yE,4i+2b+c 
with i ^ iK By Lemmas 031 we have z E ,4 l+ 2b+c(5t + 2) = 3 - 6 - c + (26 - 1)8(C{ = 0) + (2c - 
1)6 (C% = 0) for i 7^ z*. Observe that this quantity is always at least a unity. Since variables 
Vb,*i yc,* are passive, by inspecting rows E of matrix R we obtain that for all s e (0, 1) 

z E ,M+2b+c(5t + 2 + s) > 1 - (3M >3 (5t + 2 + s) - y Ai3 (5t + 2)) 

> 1-5 

>0. 



Therefore variables yE,m+2b+c remain passive. Now consider i 



v. 



In this case we have 



z E M+2b+c(5t + 2) = 2 - 6 - c + (26 - 1)6{C\ = 0) + (2c - 1)5(C| = 0). Case by case in- 
spection shows that —b + (26 — 1)6{C\ = 0) = — 1 when 6 = min(C*, 1) and > other- 
wise. Similarly — c + (2c— 1)6{C\ = 0) = — 1 when c = min(C2,l). Therefore yE,a t +2b+c 
remains passive unless 6 = mm(C{, 1) and c = min(C|, 1). Before we turn to yE,4i t +2b+c when 
6 = min(C^, 1), c = min(C2, 1), we need to analyze y F ^. 

For these variables we recall from Lemmas l2|3l that z Fj u+2b+c(5t + 2) = 4 + T(i, 6, c) > 1 for 
all i. Therefore, for s G (0, 1) 

z F ,4i+2b+c(5t + 2 + s) > 1 - (z/A, 3 (5t + 2 + s) - y A3 (5£ + 2)) = 1 - s > 0. 

We conclude that y F ^ remain passive. 

Let us return to yE,4i t +2b+c when 6 = min(Cj, l),c = min(C2, 1). Observe that in this case 
T(i,6, c) = —3. Therefore 



ZE,4i t +2b+c(5t 

Therefore yE,4i t +2b+c is active in the interval (5t + 2, 5t + 3). 



2 + s) = 3 + T(i, 6, c) - s + y£,4i'+26+ c (5t + 2 + s) - y E ,4i*+2b+c{5t + 2) 
= -S + y j B,4it+2fe+c(5t + 2 + s) - y F M+2b+c(5t + 2). 
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It is now easy to compute the state at time 5t + 3. Let 6* = min(C^, l),c* = min(C2, 1). We 
have from the definition of BE sub-matrix and from the state of the system at time 5t + 2 that 
for all s e (0, 1) and i ^ i t+l 

ZB,i(5t + 2 + s) = yE,Ai t +2bt+ c t{5t + 2 + s) - yE,4;*+3 (5t + 2) = s 

and ZB,it+i(5t + 2 + s) =0. Evaluating at s = 1, we obtain the claim for ZB,*(5t + 3). We have 
for % = 1,2 

2C,i(5£ + 2 + s) = C* + (Ai + l)(l/ J B,4it+26*+ C *(5t + 2 + s) - yE,4i t +2¥+ct(5t + 2)) 

= C* + (A, + 1) 5 . 

Evaluating at s = 1, we obtain zc,i(5t + 3) = C* + Aj + 1. 
Variables zd,* remain unchanged. 
Now we analyze ze,*- We have for Ai + 26 + c ^ Ai 1 + 26* + c* 

ZE,4i+2b+c{5t + 2 + s) = 3 + T(i, b, c) - s. 

and ^e,4j*+26*+c*(^^ + 2 + s) = 0. Recalling that T(i*, 6*, c*) = —3, we may combine two cases as 

z E ,4i+2b+c(5t + 3) = 2 + T(i, 6, c) + l{(i, 6, c) = (i*, 6*, c*)}. 

Finally, for 4i + 26 + c 7^ 4i* + 26* + c* we have 

z FAi+2 b+c{5t + 2 + s) = 4 + T(i, 6, c) - s, 

and 

z F ,4i t +2V+ c t(5t + 2 + s) = 4 + T(i\ 6*, c*) - s + s 

= 1. 

Again we may combine the cases by saying 

3F,4i+26+c(5t + 3) = 3 + T(i, 6, c) + l{(i, 6, c) = (2*, 6*, c*)}. 

This completes the proof of the lemma. D 

5.5 Interval [5t + 3, 5t + 4] 

Lemma 5. In i/ie time period (5t + 3, 5t + 4) variable da,4 is active. The variables yF,u+2b+c 
are passive in the time interval (5t + 3, 5t + 3 + (3 + T(i, b, c) + l{(i, b, c) = (i*, 6*, c*)})/4) and 
active during the time interval (5t + 3 + (3 + T(i, b, c) + 1{(«, 6, c) = («*, 6*, c*)})/4, 5t + 4). v4// the 
remaining variables are passive in (5t+3, 5t+4). The state z(5t+A) is as follows. ZB,it+i(5t+A) = 
and z Bji (5t + 4) = 1 for i ^ i t+1 , z c ,i(5t + 4) = C\ + A h i = 1, 2, «u,«(5* + 3) = 0, i = 1, 2, 
-2_e,*(5t + 4) = 3 ; and -2f>(5£ + 4) = 0. Namely, in vector form 
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z B (5t + A) 



z E {5t + 4) 



i 



i 



1 

( '■ \ 

3 

V : / 



z c (5t + 4:) 



CI 



A 1 



z D {5t + A) 



z F (5t + 4) 



/ : \ 



V : / 



Proof. Variables y E ,* are passive since BA* A = and the entries of BE are non-negative. For 
yc,* variables we have by non-negativity of CE that for s G (0, 1) 

z c ,i(St + 3 + s) > z CA (5t + 3) - {yAA^t + 3 + s) - y AA (5t + 3)) = z c ,i(5t + 3) - s > 0, 

since by Lemma H] we have Zc,i (5t + 3) = C\ + A, + 1 > 1 (recall that by feasibility of the Counter 
Machine we have C\ + Aj > 0). Thus yc,* remain passive. Variables y E ,* remain passive since 
DA iA = and all other entries in rows of R corresponding to D are non-negative. Variables y E ,* 
remain passive since EA* A = 0, variables y E ,* and yc,* are passive and all other entries in rows 
of R corresponding to E are non-negative. 

Finally, we analyze yF,* variables. Applying Lemma HJ and the fact that yB,*,yc,*,VD,* and 
y E ,* variables are passive, we have for s G (0, 1) 

ZF,4i+2b+c(5t + 3 + s) = z F ,4i+2b+c(5t + 3) - 4(y yM (5£ + 3 + s) - ?M,4(5t + 3)) 

+ (2/F,4i+2b+c(5t + 3 + s) - y F ,4i+2b+c{5t + 3)) 



3 + T(z, b, c) + l{(z, b, c) 



:t it J. 



#,<*)} 



- As + (y E ,4i+2b+c(5t + 3 + s) - yF,4i+2b+c(5t + 3)) 

It is easy to verify directly that T(i,b,c) < and in the case (i,b,c) = (i*, &*, c*), additionally 
T(i, 6, c) = -3. Therefore 3 + T(i, 6, c) + l{(i, 6, c) = (z f , 6*, c*)} < 3. It follows that 

3 + T(z, 6, c) + l{(z, 6, c) = (z*, 6 4 , c*)} - 4s 



remains positive for s < (3 + T(z, b, c) + l{(i,b, c) 

T + l{(z,6,c) = 



,6*,c*)})/4. Starting with s = (3 



(z*, 6*, c*)})/4, variables z/f,4i+26+c will start increasing at the rate 4, while 
-2F,4i+2b+c(5£ + 3 + s) remains zero. This completes the analysis of active/passive variables. 
We now compute the state at time 5t + 4. Variables z E ,* do not change over the time interval 

[5t + 3, 5t + 4]. We have for s G (0, 1) 

z c ,i(5t + 3 + s) = z c ,i{5t + 3) - (y A ,4^t + 3 + s) - y AA (5t + 3)) = z c ,i{ht + 3) - s. 

We obtain the claimed result for s = 1. We check directly that variables z E ^ do not change. For 
z E ^ variables we obtain 

Z E ,4i+2b+c(5t + 3 + s) = Z EAi+2 b+c(5t + 3) + (y F ,4i+2b+c{5t + 3 + S) - yF,4i+2b+c(5t + 3)) 

= 2 + T(z, 6, c) + l{(z, 6, c) = (z*, 6*, c*)} 

+ 4(s - (3 + T(z, b, c) + l{(z, b, c) = (z*, 6 4 , c t )})/4) + . 
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Then for s = 1 we obtain -2e,4j+2&+ c (5£ + 4) = 3 as claimed. Finally, we have already verified that 
-2F,*(5t + 4) = 0. This completes the proof of the lemma. □ 

5.6 Interval [5t + 4, bt + 5] 

Lemma 6. In the time period (5t + 4, 5£ + 5) variable y^,5 ^s active and all other variables are 
passive. The state z(5t + 5) zs as claimed in Theorem^ 

Proof. It is straightforward to verify starting from y Bif till yp t *, in this order that all of these 
variables remain passive. Thus y^,5 is the only active variable. This variable impacts only 
variables zf,*- Specifically, for s e (0, 1) 

z FAi+2b+ c(5t + 4 + s) = z FAi+2b+c (5t + 4) + 4(yA 5 (5£ + 4 + s) - y A ,s{5t + 4 )) = 4s ; 
where Lemma[5]was used to assert ZF,M+2b+c(^t + 4) = 0. Fixing s — 1, we obtain the claim. □ 
Lemmas [2HB] are combined to complete the proof of Theorem [SJ □ 

6 Modification of a Skorokhod problem and stability 

In this section we modify the Skorokhod problem in such a way that the Counter Machine enters 
a halting state i* if and only if the Skorokhod problem initialized as described in the beginning 
of Section [5] is stable. Additionally we show that the R matrix corresponding to the modified 
problem is completely-5. We then use Theorem [7J to complete the proof of our main result, 
Theorem [5j 

Thus consider the Skorokhod problem constructed in the previous section. Suppose that i* is 
the halting state. Recall by Theorem [7J that we may assume without the loss of generality that 
if the halting state i* is reached at some time T, then the values of the counters at this time are 
given as Cf = Cj = 1. We now modify the Skorokhod problem as follows. We do not change 
the vector 9. The matrix R is modified as follows. Set ABk t i* = —1 for k = 3,4,5 (in particular 
the matrix AB is no longer zero). Set EB ii+ 2b+c,i* — — 3 + b + c for alH = 1, . . . , m and 6, c = 0, 1 
(it used to be —1 for i = i* and zero for i ^ i*). Finally, let FB^ i+ 2b+ c ,i* = — 4 + 6 + c for all 
i = l,...,m and b, c = 0, 1. All the remaining entries remain the same. For convenience we 
denote the modified matrix by R again. 

Proposition 1. The matrix R is completely-S . 

Proof. Consider a vector v defined as follows: vaa = v^,2 = 1, V A,3 — v a,a — v a$ — 3, v B,i = 
2, 1 < i < m,vc t * = 5,v D ^ = 2,ve,* = 25, vp t * = 38. Consider any principal submatrix R and 
let I be the set of indices in R. We claim that Rvj > 0. Namely, we claim that the vector v 
achieves the required property for every principal submatrix R. Consider any (A, k) G I (if any 
exists). Then 

JJ R{A,k),jVj = ^2 R(A,k),jVj + V A ,k 

j£i jei,j^(A,k) 

But the only negative component in the row (A, k) is ABk.i* = —1 for k = 3,4, 5. Since vb,%* = 2 
and VA,k = 3 for 3 < k < 5 we obtain that the expression is at least 3 — 2 > 0. 



Consider any (B,i) G /. We have by non-negativity of BE 

^,R{B,i)jVj > -v AA l{(A, 1) g /} + v B ,i > -1 + 2 > 0. 

Consider any (C, z) G I. We have by non-negativity of CE 

J2 R (CAi v i > -v A ,il{{A, 1) G /} - v AA l{{A, 4) G /} + v Ctl > -1 - 3 + 5 > 0. 

Similarly for any (D,i) G / 

^ R{D,i),J V J > ~VA,2l{{A, 2) G /} + u Ai > -1 + 2 > 0. 

For any (£", 4z + 26 + c) G I, z 7^ z* we have 

XI ^(B,4i+2frfc)jVj > ~(& + cK,ll{(A 1) G /} - VA,3l{(A, 3) G /} - UB,il{(-B, z) G /} 

- (3 - 6 - c)v Bii *l{(B, f ) G /} + (26 - l)v C) il{(C, 1) G /} 

+ (2C - 1)U C)2 1{(C, 2) G J} + V E ,4i+2b+c 

> -2 -3-2-3x2-5-5 + 24 
>0. 

The case i = i* is similar, except the term ug^l{(.B,i) G 1} disappears. Similarly, for any 

(F, M + 26 + c) G / we have 

2 J R(F,4i+2b+c),jVj 

> -(6 + c)z; Ail l{(^, 1) G /} - ^, 3 1{(A 3) G /} - 4^ 4 1{(A, 4) G /} - v B> il{(B, 2) G /} 

- (4 - 6 - c^lftS, z*) G /} + (26 - l)vc,il{(C, 1) G /} 

+ (2c - l)i> Ci2 l{(C, 2) G 1} + v FAi+ 2b+c 

> -2 -3-4x3-2-4x2-5-5 + 38 

>0. 

□ 

Next we connect the halting property of the Counter Machine with stability of the Skorokhod 
problem. Introduce a new notation. Given a state i and counter values Ci, C 2 , the corresponding 
state of the Skorokhod problem described in the beginning of Section [5] is denoted by z h 1} 2 . In 
particular z%J'° 2 = 0, z^'° 2 = 1, %' ^ z, and 4?' C2 = C« * = 1, 2 - 

Proposition 2. Consider a Counter Machine starting in configuration (i°, zj 1 , 0°) o,nd the corre- 
sponding Skorokhod problem (9,R) initiated in state z(0) = z % ,Zl ' Z2 . Then the Counter Machine 
enters state i* if and only if the Skorokhod problem (z(0),9,R) is stable. Specifically, if the 
Counter Machine enters state i* in time T , then z(5T + 1) = 0, and if the Counter Machine 
never reaches state i* then z(t) 7^ for all t. 
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Proof. Assume first that the halting state i* is never reached, and let us show that (z(0),9,R) 
is not stable. For this goal let us show that the dynamics of y(t),z(t) evolve exactly in the 
same way as in the case before augmentation. The proof is by induction in t. The base case 
is covered by the initialization of our Skorokhod problem. Suppose the assertion holds for all 
t' < 51 In particular, z(5t) = z % A A. We now show that it holds over time [5t,5t + 5]. In 
particular, z(ht + 5) = z % ,Cl ,c ? . Consider first the interval [5t, 5t + 1]. Observe that i/a,i 
increases at most the unit rate over the interval (5t, 5t + 1), since all the entries in the row (A, 1) 
are non-negative. Since i* ^ i l then ZB,i*(5t) = 1. Also zs,i* can decrease at the rate at most 
the rate of increase of i/a,i (as all other entries in the row (B,i*) of R are non-negative. It 
follows that zb,i* remains positive in the interval (5t, 5t+ 1) and therefore, ys.i* is passive. Since 
the only change in the matrix R has to do with the column variable (B, i*), then repeating the 
argument of Lemma [T] and Lemma [2] we obtain that the dynamics of y(t), z(t) is the same as 
before augmentation. In particular the state z(5t + 1) is described in Lemma [2J 

Now consider interval (5t + l, 5t+2). First consider (5t+l, 5t + 3/2). Recall that yA,i increases 
at most the unit rate. It then follows that ys,i* also can increase at most the unit rate since 
BAi*^ = — 1 and all the other entries in the row {B,i*) are non-negative. Let us show that over 
this interval yA,k are passive for k = 3,4,5. Their value at time 5t + 1 is 1. They can decrease 
at most the rate —2, with —1 due to 0A,k = — 1 and another —1 due to the increase rate of ys,i* 
being bounded by one. Thus yA,k, k = 3, 4, 5 are passive in (5t + 1, 5t + 3/2). This further implies 
that yA,2 increases at the unit rate over this interval (to compensate 9a,2 — ~ !)• But then as 
in the proof of Lemma [I] we conclude that yA,i remains passive. This further implies that ys,i* 
remains passive. We conclude that over the time interval (5t + l,5t + 3/2) the system evolves 
exactly in the same way as the system before modification, namely as described in Lemma El 
In particular ZA,k — 1 for k = 3,4, 5. We now repeat the same argument as for the interval 
(5t + 1, 5£ + 3/2) to argue that over the time interval (5t + 3/2, 5t + 2) the dynamics is the same 
as before the modification. 

Now consider interval (5t + 2,5t + 3). Since ZA,i(5t + 2) = 1, then yA,i remains passive in 
this time interval. Then ys,%* remains passive in this time interval as well. Then the dynamics 
of y and z is the same as before the augmentation and is described in Lemma SJ In particular 
ZB,i*(5t+3) = 1 (since i* ^ i r ). Recall from Lemmas |5] and |6]that zb^s) = 1 for s = (5t+3, 5t+5) 
and i ^ i t+1 . By assumption i* ^ i t+1 . Therefore this applies to zb^* as well and in particular 
the dynamics over (5t + 3, 5t + 5) is exactly as before the augmentation. This completes the 
proof Proposition |2] for the case when the state i* is never reached. 

Now assume that the halting state i* is reached in time t = T. We recall from Theorem [7] 
that then at time T both counters are equal to unity. Repeating the previous argument we have 
that z(5T) = z 1 ' 1 ' 1 . The claim of Proposition |2] follows immediately from the following lemma. 

Lemma 7. Over the time interval (5T, 5T + 1) the variables yA,i and yB,i* are active at the unit 
rate. All the remaining variables are passive. Moreover z(5T + 1) = 0. 

Proof. We have 

za,2(5T + s) > z A ,2(5T) - s = 1 - s. 

It follows that yA,2 is a passive variable in (5T, 5T + 1). 

For variables yB,*, zb,* we have for i ^ i* and by non-negativity of BE sub-matrix that 

ZB,i(5T + s) > z B ,i{hT) - Gm,i(5T + s) - y A ,i(5T)) = 1 - Gm,i(5T + s) - y A ,i(5T)) 
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Recall that y A> i increases at most the unit rate. This means z Bt i{5T + s) > for all s G (0, 1) 
and therefore y B ^ remains passive. We will show later that y B ,i* is active. For now let us obtain 
a partial relation on this variable. We have 

z B ,i*(5T + s) = -(y A ,i(5T + s) - y A ,i(5T)) + y Bji * (5T + s) -y B ,i*(5T) 

+ J2 BEi.j(y EiJ (5T + s)- y E>j (5T)). 
i 

From the non-negativity of BE submatrix and applying constraint ([3]) we obtain that 

y B ,i*(5T + s)- y B ,i*(5T) < y AA (5T + s) - y A i(5T), (18) 

for every s G (0, 1) (otherwise there exist an interval over which y B ^* is active while z Bj i* is 
positive). In particular, y Bti * increases at most the unit rate since so does y AA . 
Now consider variables y A<3 , z A $. We have applying (TT8|) 



z A3 (5T + s) > 1 - s + y A>1 (5T + s) - y AA (5T) - {y B>i *{5T + s) - y B>i * (5T)) > 1 - s > 

for s G (0, 1). It follows that y A $ is passive in (5T, 5T + 1). For a similar reason y AA remains 
passive as well. We will return to y A *> later. 

Recall that z[ — z\ — 1. This means zc,k(5T) — 1, k — 1, 2. Using a similar argument as for 
yB,i,i 7^ i*, non- negativity of CE and the fact that y AA is passive, we establish that yc,k remain 
passive. Variables yo,k,k = 1,2 remain passive since y A ,2 is passive as we established, and all 
other entries in the D rows of R are non-negative. 

For E variables, recall that y A)3 , y Bti , i ^ i* and yc,* are passive. Then 

z E Ai+2b+c{5T + s) =3- (b+c)(y AA (5T + s) - y AA (5T)) - (3 - b - c) (y B>i * (5T + s) -y Bji *(5T)) 

+ yE,4i+2b+c(5T + S) - yE,4i+2b+c(5T) + yF,4i+2b+c(5T + S) - yFAi+2b+c(5T) 

Recall that y AA and y B ,i* increase at most the unit rate. We see that 

ZE A i+2b+c > 3 - (b + c)s - (3 - b - c)s = 3 - 3s 

and therefore remains positive in (5T, 5T + s). Thus y E ,* variables are passive. For a similar 
reason and since in addition y AA is passive, as we have established, variables y E ,* are passive as 
well. 

Let let us consider variable y B ^*. In light of the fact that y E ,* are passive, we have 

zu,i'(5T + s) = -(y AA (5T + s) - y AA (5T)) + (y B ^(5T + s) - y B ^(5T)). 

This by constraint ([3]) implies 

VbA^ t + s)~ VbA^T) = Vaa^T + s)- y AA (5T) (19) 

for all s G (0, 1). Now let us return to y AA . Since y A ,2,yA,3,yA A are passive then 

z AA (5T + s) = -s + y AA {5T + s) - y AA (5T), 

which implies that y AA is active at unit rate and the same holds for y Bji * by (TT9~j) . 
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Finally, turning to y A ^ we have 

z At5 (5T + s) = -s + 2(y A>1 {5T + s) - y A ,i( 5T )) + 2M, 5 (5T + s) - 2M, 5 (5T) 
-(j/B,i*(5T + s)-y fl>i .(5T)) 
= -s + 2s + y A *>{hT + s) - y A , 5 (5T) - s 
= 2/a,5(5T + s) -y A5 (5T). 

This implies that y A *> is passive in the interval (5T, 5T + 1). 

We have established that y At i and ys,%* are the only active variables and they increase at the 
unit rate. With this in mind it is straightforward to verify that z(5T + 1) = 1. This completes 
the proof of the lemma. □ 

This completes the proof of Proposition [2j □ 

Acknowledgements 

We are delighted to acknowledge many enlightening discussions with Maury Bramson. 

References 

[AB89] A. El Kharroubi A. Bernard, Reflexions (ou regulations) de processus dans le premier 
"orthant" de R n , C. R. Acad. Sci. Paris Ser. I Math. 309 (1989), 371-375. 

[AB91] , Regulations deterministes et stochastiques dans le premier "orthant" de R n , 

Stochastics Stochastics Rep. 34 (1991), 149-167. 

[AEKOO] A. Yaacoubi A. El Kharroubi, A. Ben Tahar, Sur la recurrence positive du mouvement 
brownien reflechi dans I 'orthant positif de R n , Stochastics Stochastics Rep. 68 (2000), 
229-253. 

[AEK02] A. Yaacoubi Abdelhak A. El Kharroubi, A. Ben Tahar, On the stability of the linear 
Skorohod problem in an orthant, Math. Methods Oper. Res. 56 (2002), 243-258. 

[BBK + 01] V. D. Blondel, O. Bournez, P. Koiran, C. H. Papadimitriou, and J. N. Tsitsiklis, 
Deciding stability and mortality of piecewise afflne systems, Theoretical Computer 
Science 225 (2001), no. 1-2, 687-696. 

[BDH] M. Bramson, J. G. Dai, and J. M. Harrison, Positive recurrence of reflecting Brownian 
motion in three dimensions, Preprint. 

[BL09] A. Budhiraja and C. Lee, Stationary distribution convergence for generalized Jackson 
networks in heavy traffic, Math. Oper. Res. 34 (2009), 45-56. 

[Bra] M. Bramson, A positive recurrent reflecting Brownian motion with divergent fluid 

path, Preprint. 

[BTOOa] V. D. Blondel and J. N. Tsitsiklis, The boundedness of all products of a pair of matrices 
is undecidable, Systems and control letters 41 (2000), no. 2, 135-140. 



22 



[BTOOb] , A survey of computational complexity results in systems and control, Auto- 

matica 36 (2000), no. 9, 1249-1274. 

[CY01] H. Chen and D. Yao, Fundamentals of queueing networks: Performance, asymptotics 
and optimization, Springer- Verlag, 2001. 

[DW94] P. Dupuis and R. J. Williams, Lyapunov functions for semimartingale reflecting Brow- 
nian motions, Annals of Probability 22 (1994), 680-702. 

[Gam02] D. Gamarnik, On deciding stability of constrained homogeneous random walks and 
queueing systems, Mathematics of Operations Research 27 (2002), no. 2, 272-293. 

[Gam07] , Computing stationary probability distribution and large deviations rates for 

constrained homogeneous random walks. The undecidability results, Mathematics of 
Operations Research 27 (2007), no. 2, 272-293. 

[GK07] D. Gamarnik and D. Katz, Correlation decay and deterministic FPTAS for counting 
list- colorings of a graph, Proceedings of 18th ACM-SIAM Symposium on Discrete 
Algorithms (SODA), 2007. 

[GKR09] D. Gamarnik and D. Katz-Rogozhnikov, On deciding stability of queueing networks 
under priority scheduling policy, Annals of Applied Probability 19 (2009), 2008-2037. 

[GS10] C. Goodman-Strauss, Cant decide? undecidel, Notices of the American Mathematical 
Society 57 (2010), 343-356. 

[GZ06] D. Gamarnik and A. Zeevi, Validity of heavy traffic steady-state approximations in 
open queueing networks, Ann. Appl. Prob. 16 (2006), no. 1, 56-90. 

[Har90] J. M. Harrison, Brownian motion and stochastic flow systems, Krieger Publishing 
Company, 1990. 

[H0066] P. Hooper, The undecidability of the Turing machine immortality problem, The Jour- 
nal of Symbolic Logic 2 (1966), 219-234. 

[HU69] J. Hopcroft and J. Ullman, Formal languages and their relation to automata, Addison- 
Wesley. Boston, MA, 1969. 

[Sip97] M. Sipser, Introduction to the theory of computability, PWS Publishing Company, 
1997. 

[TW93] L. M. Taylor and R. J. Williams, Existence and uniqueness of semimartingale re- 
flecting Brownian motions in an orthant, Probab. Theory Related Fields 96 (1993), 
283-317. 

[Wil95] R. J. Williams, Semimartingale reflecting Brownian motions in the orthant, Stochastic 
Networks, IMA Volumes in Mathematics and Its Applications, Volume 71, eds. F. P. 
Kelly and R. J. Williams. Springer- Verlag, New York, 1995, pp. 125-137. 



23 



